docs: dogfood report for v3.10.1-dev.80#1180
Conversation
|
Claude finished @carlos-alm's task —— View job PR Review: Dogfood Report v3.10.1-dev.80Review Progress
Root Cause AnalysisProblem Being Solved: This PR adds comprehensive dogfooding documentation for codegraph v3.10.1-dev.80, providing validation that 12 newly-ported native extractors achieve byte-identical parity with WASM engines and documenting 4 critical bugs discovered during testing. Why This Approach: Documentation-only PR that follows the established pattern in Risk Assessment: Zero risk - pure documentation with no code changes. Backlog Compliance
Critical ConcernsNone identified. This is exemplary documentation: Strengths
Quality Indicators
Content ValidationThe report demonstrates the exact kind of quality validation codegraph needs:
Value to ProjectThis documentation:
Final Recommendation
|
Greptile SummaryThis PR updates the dogfooding report for
Confidence Score: 5/5Safe to merge — docs-only change with no code modifications. The change touches a single markdown report file, correcting two prose count references (12→11) that now agree with every other count in the document, and appending one tracking row to the issues table. No logic, schema, or executable code is affected. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Dogfood Session\n2026-05-20] --> B[§5 Engine Comparison\n11 newly-ported extractors]
A --> C[§8 Embedding Benchmark\nminilm / jina-small / jina-base†]
B --> D[§6 Release-Specific Tests\n11 extractor ports #1097–#1107]
B --> E[§12 Overall Assessment\n11 new native extractors]
C --> F[jina-base run deferred]
F --> G[Issue #1181 filed\npost-session]
G --> H[Benchmark backfilled\ninto §8 with ± tolerance note]
H --> I[§13 Issues & PRs table\n#1181 added as Issue^1]
D -.->|count fix: 12→11| D2[✓ Corrected in this PR]
E -.->|count fix: 12→11| E2[✓ Corrected in this PR]
I -.->|row added| I2[✓ Added in this PR]
Reviews (11): Last reviewed commit: "fix: resolve merge conflicts with main" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| ## 6. Release-Specific Tests (changes since v3.10.0) | ||
|
|
||
| The v3.10.1-dev.80 series adds 12 native extractor ports (#1097–#1107), several language-specific fixes (#1109, #1122, #1123, #1124, #1127, #1128, #1158), a MCP enhancement (#1149), benchmark-stability fixes (#1119, #1120, #1131, #1133, #1134), and CI hardening (#1146, #1151, #1164). |
There was a problem hiding this comment.
The PR description states "12 newly-ported native extractors validated," but the report consistently counts 11 throughout — the §5 table lists 11 languages, the §5 conclusion reads "all 11 newly-ported extractors," and the §6 changelog entry enumerates 11 ports (#1097–#1107). One of these two numbers is wrong and will cause confusion when the report is referenced later. If the correct count is 11, the PR description should be updated to match; if it's 12, the missing language should be added to the §5 table and the §6 test row.
|
|
||
| - No regressions vs the v3.10.0 baseline in `generated/benchmarks/BUILD-BENCHMARKS.md`. The corpus shrank (745 → 612 files) due to PR #1134's fixture exclusion, but per-file metrics improved on every engine. | ||
| - Native fast-skip preflight (#1054) is firing as expected: 16 ms no-op rebuild matches WASM's, validating the `detectNoChanges` short-circuit. | ||
| - The 1-file rebuild gap (WASM 45ms vs Native 67ms) is the inverse of full-build performance — WASM's lighter orchestrator setup wins on tiny incremental work. |
There was a problem hiding this comment.
jina-base embedding benchmark published as incomplete
The embedding benchmark table has jina-base (768d) with the value _benchmark still running at report cut_. Publishing a report with a known-pending data point makes the §8 table misleading — reviewers cannot assess whether jina-base regressed, improved, or is even viable for the release. Either wait for the run to finish and fill in the numbers, or explicitly mark this row as "not completed, see follow-up" and omit the empty columns so it's clear no data was collected rather than data being redacted.
There was a problem hiding this comment.
Fixed in ca4146c. Replaced the "benchmark still running at report cut" placeholder with an explicit "not completed in this session" marker linked to follow-up issue #1181, and zeroed out the empty Hit@k columns with em-dashes so it's clear no data was collected (rather than redacted). The follow-up issue tracks finishing the jina-base run and backfilling the numbers.
Fix two Greptile P1/P2 findings in the v3.10.1-dev.80 dogfood report: - §6 changelog said "12 native extractor ports (#1097-#1107)" but the PR range and the §5 parity table both enumerate 11. Correct to 11 so the body is internally consistent (the §5 table and conclusion already read 11/11). - §8 embedding-benchmark table left the jina-base (768d) row as "benchmark still running at report cut". Replace the placeholder with an explicit "not completed in this session" note and link to the follow-up issue #1181 so the missing data is clearly tracked rather than reading as redacted.
|
Fixed in 6d9e1bb. The §12 Overall Assessment opening sentence at line 435 now reads "11 new native extractors" — matching the §5 conclusion, §6 intro, §11 testing plan, and the §12 bullet list. The report is now fully self-consistent. |
|
FYI: opened #1186 to backfill the actual jina-base numbers for §8, which replaces the same placeholder line this PR clarifies. If #1186 merges first, the §8 edit here becomes a no-op — the line that says No action needed from this PR yet; just flagging the overlap so whoever lands first knows the other branch will need a rebase / trivial conflict resolution. |
|
Addressed the §13 follow-up note from the latest Greptile review:
Commit: 42ed9a4 |
1 participant
Summary
Adds the dogfooding report for
@optave/codegraph@3.10.1-dev.80(run on 2026-05-20).Test plan
fix/1175-embed-install-cwdontoorigin/main(separates docs from the code fix in fix(embed): install @huggingface/transformers into codegraph's host node_modules #1178 per one-PR-one-concern)